Intra/inter mode decision for predictive frame encoding

ABSTRACT

This invention predicts that intra mode prediction is more effective for the macroblocks where motion estimation in inter mode prediction fails. This failure is indicated by a large value of the inter mode SAD. This invention performs intra mode prediction for only macro blocks have larger inter mode SADs. The definition of a large inter mode SAD differs for different content. This invention compares the inter mode SAD of a current macroblock with an adaptive threshold. This adaptive threshold depends on the average and variance of the SADs of the previous predicted frame. An adaptive threshold is calculated for each new predictive frame.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/228,350, filed on Dec. 20, 2018, which is a continuation of U.S.patent application Ser. No. 15/419,512 filed on Jan. 30, 2017 (now U.S.Pat. No. 10,165,270), which is a continuation of U.S. patent applicationSer. No. 12/751,211 filed on Mar. 31, 2010 (now U.S. Pat. No.9,560,350), which claims priority to U.S. Provisional Application No.61/165,213 filed Mar. 31, 2009, all of which are incorporated herein byreference.

TECHNICAL FIELD

The technical field of this invention is video encoding.

BACKGROUND

Spatial domain intra mode prediction an important tool in advanced videocoding standards such as H.264 and AVS which improve coding efficiencyover older standards. The H.264 intra prediction allows three modes,intra 16×16 (I16×16), intra 8×8 (I8×8) and intra 4×4 (I4×4) based on theprediction block size. For a predictive frame the encoder decides toencode a macroblock in the intra mode when motion estimation and theinter mode prediction fail to find a good match from the referenceframe. The encoder computes a measure of similarity for both the bestintra prediction mode and the best inter prediction mode. This measureof similarity is often the sum of absolute differences (SAD). Theencoder then selects the mode with minimum SAD (best similarity). Thecomputation of SAD for best intra prediction mode is not trivial andrequires several arithmetic operations to search through all the allowedintra prediction modes. When using dedicated memory based hardwarearchitecture with different hardware accelerators for intra mode andinter mode prediction, there will be additional transfer of data fromexternal memory to hardware accelerator internal memory to compute intraprediction SAD. Switching off intra mode search completely results innoticeable video quality artifacts. Thus there is a need in the art toreduce the computational complexity and data transfer needs of intraprediction.

SUMMARY OF THE INVENTION

Statistical data shows intra mode macroblock coding is best when theinter mode macroblock SAD is very high. This invention uses thiscondition. Rather than computing the intra mode SAD for all macroblocksof a predictive frame, this invention computes the intra mode SAD foronly some macroblocks. The inter mode SAD computation should bepipelined before intra mode SAD computation to realize the bestperformance gain.

For macroblocks where the inter mode SAD is smaller than a predeterminedconstant, the intra mode SAD is not computed. Such macroblocks are codedin the inter mode. For other macroblocks both the intra mode SAD and theinter mode SADs are computed. The encoder selects the encoding typebased on comparison of these SAD computations.

The constant FastIntraThreshold is calculated from the macroblock prepixel inter mode SAD averaged over the entire frame and its variance.The average and variance from previous predictive frame are used in thecalculation for the current frame. The average and variance of thecurrent frame would not be known until entire frame is encoded. Thuscalculating FastIntraThreshold based upon the current frame introduces a“chicken-egg” problem.

This calculation of FastIntraThreshold includes a quality factor.Increasing the quality factor increases the proportion of macroblockswhich are intra mode searched. This provides better quality but resultsin smaller computational reduction. The preferred embodiment includessix possible quality factor levels used in varying image conditions.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other aspects of this invention are illustrated in thedrawings, in which:

FIG. 1 illustrates the organization of a typical digital signalprocessor to which this invention is applicable (prior art);

FIG. 2 illustrates details of a very long instruction word digitalsignal processor core suitable for use in FIG. 1 (prior art);

FIG. 3 illustrates the pipeline stages of the very long instruction worddigital signal processor core illustrated in FIG. 2 (prior art);

FIG. 4 illustrates the instruction syntax of the very long instructionword digital signal processor core illustrated in FIG. 2 (prior art);

FIG. 5 illustrates an overview of the video encoding process of theprior art;

FIG. 6 is a flow chart of process for determining inter or intra modeprediction for a particular frame.

DETAILED DESCRIPTION

FIG. 1 illustrates the organization of a typical digital signalprocessor system 100 to which this invention is applicable (prior art).Digital signal processor system 100 includes central processing unitcore 110. Central processing unit core 110 includes the data processingportion of digital signal processor system 100. Central processing unitcore 110 could be constructed as known in the art and would typicallyincludes a register file, an integer arithmetic logic unit, an integermultiplier and program flow control units. An example of an appropriatecentral processing unit core is described below in conjunction withFIGS. 2 to 4.

Digital signal processor system 100 includes a number of cache memories.FIG. 1 illustrates a pair of first level caches. Level one instructioncache (L1I) 121 stores instructions used by central processing unit core110. Central processing unit core 110 first attempts to access anyinstruction from level one instruction cache 121. Level one data cache(L1D) 123 stores data used by central processing unit core 110. Centralprocessing unit core 110 first attempts to access any required data fromlevel one data cache 123. The two level one caches are backed by a leveltwo unified cache (L2) 130. In the event of a cache miss to level oneinstruction cache 121 or to level one data cache 123, the requestedinstruction or data is sought from level two unified cache 130. If therequested instruction or data is stored in level two unified cache 130,then it is supplied to the requesting level one cache for supply tocentral processing unit core 110. As is known in the art, the requestedinstruction or data may be simultaneously supplied to both therequesting cache and central processing unit core 110 to speed use.

Level two unified cache 130 is further coupled to higher level memorysystems. Digital signal processor system 100 may be a part of amultiprocessor system. The other processors of the multiprocessor systemare coupled to level two unified cache 130 via a transfer request bus141 and a data transfer bus 143. A direct memory access unit 150provides the connection of digital signal processor system 100 toexternal memory 161 and external peripherals 169.

FIG. 2 is a block diagram illustrating details of a digital signalprocessor integrated circuit 200 suitable but not essential for use inthis invention (prior art). The digital signal processor integratedcircuit 200 includes central processing unit 1, which is a 32-biteight-way VLIW pipelined processor. Central processing unit 1 is coupledto level one instruction cache 121 included in digital signal processorintegrated circuit 200. Digital signal processor integrated circuit 200also includes level one data cache 123. Digital signal processorintegrated circuit 200 also includes peripherals 4 to 9. Theseperipherals preferably include an external memory interface (EMIF) 4 anda direct memory access (DMA) controller 5. External memory interface(EMIF) 4 preferably supports access to supports synchronous andasynchronous SRAM and synchronous DRAM. Direct memory access (DMA)controller 5 preferably provides 2-channel auto-boot loading directmemory access. These peripherals include power-down logic 6. Power-downlogic 6 preferably can halt central processing unit activity, peripheralactivity, and phase lock loop (PLL) clock synchronization activity toreduce power consumption. These peripherals also include host ports 7,serial ports 8 and programmable timers 9.

Central processing unit 1 has a 32-bit, byte addressable address space.Internal memory on the same integrated circuit is preferably organizedin a data space including level one data cache 123 and a program spaceincluding level one instruction cache 121. When off-chip memory is used,preferably these two spaces are unified into a single memory space viathe external memory interface (EMIF) 4.

Level one data cache 123 may be internally accessed by centralprocessing unit 1 via two internal ports 3 a and 3 b. Each internal port3 a and 3 b preferably has 32 bits of data and a 32-bit byte addressreach. Level one instruction cache 121 may be internally accessed bycentral processing unit 1 via a single port 2 a. Port 2 a of level oneinstruction cache 121 preferably has an instruction-fetch width of 256bits and a 30-bit word (four bytes) address, equivalent to a 32-bit byteaddress.

Central processing unit 1 includes program fetch unit 10, instructiondispatch unit 11, instruction decode unit 12 and two data paths 20 and30. First data path 20 includes four functional units designated L1 unit22, S1 unit 23, M1 unit 24 and D1 unit 25 and 16 32-bit A registersforming register file 21. Second data path 30 likewise includes fourfunctional units designated L2 unit 32, S2 unit 33, M2 unit 34 and D2unit 35 and 16 32-bit B registers forming register file 31. Thefunctional units of each data path access the corresponding registerfile for their operands. There are two cross paths 27 and 37 permittingaccess to one register in the opposite register file each pipelinestage. Central processing unit 1 includes control registers 13, controllogic 14, and test logic 15, emulation logic 16 and interrupt logic 17.

Program fetch unit 10, instruction dispatch unit 11 and instructiondecode unit 12 recall instructions from level one instruction cache 121and deliver up to eight 32-bit instructions to the functional unitsevery instruction cycle. Processing occurs simultaneously in each of thetwo data paths 20 and 30. As previously described each data path hasfour corresponding functional units (L, S, M and D) and a correspondingregister file containing 16 32-bit registers. Each functional unit iscontrolled by a 32-bit instruction. The data paths are further describedbelow. A control register file 13 provides the means to configure andcontrol various processor operations.

FIG. 3 illustrates the pipeline stages 300 of digital signal processorcore 110 (prior art). These pipeline stages are divided into threegroups: fetch group 310; decode group 320; and execute group 330. Allinstructions in the instruction set flow through the fetch, decode, andexecute stages of the pipeline. Fetch group 310 has four phases for allinstructions, and decode group 320 has two phases for all instructions.Execute group 330 requires a varying number of phases depending on thetype of instruction.

The fetch phases of the fetch group 310 are: Program address generatephase 311 (PG); Program address send phase 312 (PS); Program accessready wait stage 313 (PW); and Program fetch packet receive stage 314(PR). Digital signal processor core 110 uses a fetch packet (FP) ofeight instructions. All eight of the instructions proceed through fetchgroup 310 together. During PG phase 311, the program address isgenerated in program fetch unit 10. During PS phase 312, this programaddress is sent to memory. During PW phase 313, the memory read occurs.Finally during PR phase 314, the fetch packet is received at CPU 1.

The decode phases of decode group 320 are: Instruction dispatch (DP)321; and Instruction decode (DC) 322. During the DP phase 321, the fetchpackets are split into execute packets. Execute packets consist of oneor more instructions which are coded to execute in parallel. During DPphase 322, the instructions in an execute packet are assigned to theappropriate functional units. Also during DC phase 322, the sourceregisters, destination registers and associated paths are decoded forthe execution of the instructions in the respective functional units.

The execute phases of the execute group 330 are: Execute 1 (E1) 331;Execute 2 (E2) 332; Execute 3 (E3) 333; Execute 4 (E4) 334; and Execute5 (E5) 335. Different types of instructions require different numbers ofthese phases to complete. These phases of the pipeline play an importantrole in understanding the device state at CPU cycle boundaries.

During E1 phase 331, the conditions for the instructions are evaluatedand operands are read for all instruction types. For load and storeinstructions, address generation is performed and address modificationsare written to a register file. For branch instructions, branch fetchpacket in PG phase 311 is affected. For all single-cycle instructions,the results are written to a register file. All single-cycleinstructions complete during the E1 phase 331.

During the E2 phase 332, for load instructions, the address is sent tomemory. For store instructions, the address and data are sent to memory.Single-cycle instructions that saturate results set the SAT bit in thecontrol status register (CSR) if saturation occurs. For single cycle 16by 16 multiply instructions, the results are written to a register file.For M unit non-multiply instructions, the results are written to aregister file. All ordinary multiply unit instructions complete duringE2 phase 322.

During E3 phase 333, data memory accesses are performed. Any multiplyinstruction that saturates results sets the SAT bit in the controlstatus register (CSR) if saturation occurs. Store instructions completeduring the E3 phase 333.

During E4 phase 334, for load instructions, data is brought to the CPUboundary. For multiply extension instructions, the results are writtento a register file. Multiply extension instructions complete during theE4 phase 334.

During E5 phase 335, load instructions write data into a register. Loadinstructions complete during the E5 phase 335.

FIG. 4 illustrates an example of the instruction coding of instructionsused by digital signal processor core 110 (prior art). Each instructionconsists of 32 bits and controls the operation of one of the eightfunctional units. The bit fields are defined as follows. The creg field(bits 29 to 31) is the conditional register field. These bits identifywhether the instruction is conditional and identify the predicateregister. The z bit (bit 28) indicates whether the predication is basedupon zero or not zero in the predicate register. If z=1, the test is forequality with zero. If z=0, the test is for nonzero. The case of creg=0and z=0 is treated as always true to allow unconditional instructionexecution. The creg field is encoded in the instruction opcode as shownin Table 1.

TABLE 1 Conditional creg z Register 31 30 29 28 Unconditional 0 0 0 0Reserved 0 0 0 1 B0 0 0 1 z B1 0 1 0 z B2 0 1 1 z A1 1 0 0 z A2 1 0 1 zA0 1 1 0 z Reserved 1 1 1 xNote that “z” in the z bit column refers to the zero/not zero comparisonselection noted above and “x” is a don't care state. This coding canonly specify a subset of the 32 registers in each register file aspredicate registers. This selection was made to preserve bits in theinstruction coding.

The dst field (bits 23 to 27) specifies one of the 32 registers in thecorresponding register file as the destination of the instructionresults.

The scr2 field (bits 18 to 22) specifies one of the 32 registers in thecorresponding register file as the second source operand.

The scr1/cst field (bits 13 to 17) has several meanings depending on theinstruction opcode field (bits 2 to 12). The first meaning specifies oneof the 32 registers of the corresponding register file as the firstoperand. The second meaning is a 5-bit immediate constant. Depending onthe instruction type, this is treated as an unsigned integer and zeroextended to 32 bits or is treated as a signed integer and sign extendedto 32 bits. Lastly, this field can specify one of the 32 registers inthe opposite register file if the instruction invokes one of theregister file cross paths 27 or 37.

The opcode field (bits 2 to 12) specifies the type of instruction anddesignates appropriate instruction options. A detailed explanation ofthis field is beyond the scope of this invention except for theinstruction options detailed below.

The s bit (bit 1) designates the data path 20 or 30. If s=0, then datapath 20 is selected. This limits the functional unit to L1 unit 22, S1unit 23, M1 unit 24 and D1 unit 25 and the corresponding register file A21. Similarly, s=1 selects data path 20 limiting the functional unit toL2 unit 32, S2 unit 33, M2 unit 34 and D2 unit 35 and the correspondingregister file B 31.

The p bit (bit 0) marks the execute packets. The p-bit determineswhether the instruction executes in parallel with the followinginstruction. The p-bits are scanned from lower to higher address. If p=1for the current instruction, then the next instruction executes inparallel with the current instruction. If p=0 for the currentinstruction, then the next instruction executes in the cycle after thecurrent instruction. All instructions executing in parallel constitutean execute packet. An execute packet can contain up to eightinstructions. Each instruction in an execute packet must use a differentfunctional unit.

FIG. 5 illustrates the encoding process 500 of video encoding accordingto the prior art. Many video encoding standards use similar processessuch as represented in FIG. 5. In accordance with the known art adigital signal processor such as described above in conjunction withFIGS. 1 to 4 is programmed to implement encoding process 500. Encodingprocess 500 begins with input of the n th (current) frame F_(n) 501.This entire input frame is stored in a corresponding memory or set ofmemory locations. The result of encoding process 500 is the output ofencoded frame data 530 from entropy encoder 505. Frequency transformblock 502 transforms a macroblock of the pixel data into the spatialfrequency domain. This typically involves a discrete cosine transform(DCT). This frequency domain data is quantized in quantization block503. This quantization typically takes into account the range of datavalues for the current macroblock. Thus differing macroblocks may havediffering quantizations. In accordance with the H.264 standard, in thebase profile the macroblock data may be arbitrarily reordered viareorder block 504. This reordering is reversed upon decoding. Othervideo encoding standards and the H.264 main profile transmit data forthe macroblocks in strict raster scan order. The quantized data isencoded by entropy encoding block 505. Entropy encoding employs fewerbits to encode more frequently used symbols and more bits to encode lessfrequency used symbols. This process reduces the amount of encoded thatmust be transmitted and/or stored. The resulting entropy encoded data isthe output encoded frame data 530.

Video encoding standards typically permit two types of predictions. Ininter-frame prediction, data is compared with data from thecorresponding location of another frame. In intra-frame prediction, datais compared with data from another location in the same frame.

For inter prediction, data from n−1 th (previous) frame F_(n-1) 510 anddata from the n th frame F 501 supply motion estimation block 511.Motion estimation block 511 determines via a search the positions andmotion vectors of moving objects within the picture. This motion data issupplied to motion compensation block 512 along with data from n−1 thframe F_(n-1) 510. The resulting motion compensated frame data isselected by switch 513 for application to subtraction unit 506.Subtraction unit 506 subtracts the inter prediction data from switch 513from the input frame data from n th frame F 501. Thus frequencytransform block 502, quantization block 503, reorder block 504 andentropy encoding block 505 encode the differential data rather than theoriginal frame data. Assuming there is relatively little change fromframe to frame, this differential data has a smaller magnitude than theraw frame data. Thus this can be expressed in fewer bits contributing todata compression. This is true even if motion estimation block 511 andmotion compensation block 512 find no moving objects to code. If the nth frame F_(n) and the n−1 th frame F_(n-1) are identical, thesubtraction unit 506 will produce a string of zeros for data. This datastring can be encoded using few bits.

The second type of prediction is intra prediction. Intra predictionpredicts a macroblock of the current frame from another macroblock ofthe current frame. Inverse quantization block 520 receives the quantizeddata from quantization block 503 and substantially recovers the originalfrequency domain data. Inverse frequency transform block 521 transformsthe frequency domain data from inverse quantization block 520 back tothe spatial domain. This spatial domain data supplies one input ofaddition unit 522, whose function will be further described. Encodingprocess 500 includes choose intra predication unit 514 to determinewhether to implement intra prediction. Choose intra prediction unit 514receives data from n th frame F 501 and the output of addition unit 522.Choose intra prediction unit 514 signals intra predication unit 515,which also receives the output of addition unit 522. Switch 513 selectsthe intra prediction output for application to the subtraction input ofsubtraction unit 506 and an addition input of addition unit 522. Intraprediction is based upon the recovered data from inverse quantizationblock 520 and inverse frequency transform block 521 in order to bettermatch the processing at decoding. If the encoding used the originalframe, there might be drift between these processes resulting in growingerrors. This invention concerns the operation of choose intra predictionunit 514.

Video encoders typically periodically transmit unpredicted frames. Insuch an event the predicted frame is all 0's. Subtraction unit 506 thusproduces data corresponding to the n th frame F_(n) 501 data. Periodicunpredicted or I frames limits any drift between the transmitter codingand the receive decoding. In a video movie a scene change may producesuch a large change between adjacent frames that differential codingprovides little advantage. Transmitting an unpredicted frame upon ascene change would thus not be disadvantageous. Video coding standardstypically signal whether a frame is a predicted frame and the type ofprediction in the transmitted data stream.

Encoding process 500 includes reconstruction of the frame based uponthis recovered data. The output of addition unit 522 supplies deblockfilter 523. Deblock filter 523 smoothes artifacts created by the blockand macroblock nature of the encoding process. The result isreconstructed frame F′_(n) 524. As shown schematically in FIG. 5, thisreconstructed frame F′_(n) 524 becomes the next reference frame F_(n-1)510.

The prior art generally performs choose intra prediction 514 bycomputing the intra mode SAD and the inter mode SAD for each macroblock.Choose intra prediction 510 chooses the mode with minimum SAD valuewhich corresponds to the best similarity. This invention avoids intramode SAD calculation and search for macroblocks in cases where theinventors estimate inter mode coding will result in reduced videoquality. This prediction is based on a comparison between inter mode SADstatistics of the previous frame and inter mode SAD of the currentmacroblock.

This invention preferably avoids intra mode search for at least 50% ofmacroblocks. For dedicated memory based hardware accelerator basedsystems this invention significantly reduces data transfer bandwidth forthese macroblocks. These macroblocks require no data transfer fromexternal memory to dedicated memory of hardware accelerator. For theprogrammable DSP based systems this means reduction in DSP cyclesbecause intra 4×4 and intra 8×8 modes are a significant portion ofoverall encoding cycles. This invention also reduces power consumed bythe encoder. This invention is an important factor in achieving realtime encoding at 30 frames per second.

FIG. 6 illustrates program 600 which is an example of this invention.Program 600 shows the process of this invention for each video frame ofa video being encoded. The processes of program 600 generally take placewithin the choose intra prediction unit 514 except as noted below. Inthe preferred embodiment a digital signal processor such as shown abovein conjunction with FIGS. 1 to 4 is programmed to implement program 600.However, important parts of program 600 may be performed by suitablehardware accelerators. Program 600 begins with start block 601.According to this example, program 600 starts again for each new frameof the video being encoded. Test block 602 determines whether thecurrent frame is a predictive frame. If this is not the case (No at testblock 602), then this invention is not applicable. Program 600 exits atnext frame block 603. The video encoder proceeds with other processessuch as those illustrated in FIG. 5 for production of encoded frame data530.

If the current frame is a predictive frame (Yes at test block 602), thenthe current invention is applicable. Block 604 proceeds to the nextmacroblock of the current frame. When starting a new frame, this nextmacroblock is the first macroblock in the current frame. It isconventional to consider macroblocks within a particular frame in rasterscan order. Block 605 calculates a best SAD for the current macroblockin the inter mode. This corresponds to motion estimation block 511determining the motion vector and motion compensation block 512calculating the best SAD.

Test block 606 compares the best inter mode SAD with the constantFastIntraThreshold*N. In this example N is the number of pixels permacroblock, generally 256. The calculation of FastIntraThreshold inequation (5) below is a per pixel value and multiplication by N convertsthis to a per macroblock value for this comparison. In accordance withthis invention, a inter mode SAD less than FastIntraThreshold*N isexpected to correlate with a low probability that a better intra modeSAD is possible. Thus if the inter mode SAD is less thanFastIntraThreshold*N (Yes at test block 606), then this invention skipsintra mode SAD calculation. Program 600 proceeds to block 607 whichcompletes the process of inter mode encoding the current macroblock.This is equivalent to switch 513 selecting the inter position andsupplying the corresponding prediction data for the current macroblockto subtraction unit 506 and addition unit 522.

If the inter mode SAD is not less than FastIntraThreshold*N (No at testblock 606), then block 608 performs intra mode SAD calculation for thecurrent macroblock. In accordance with this invention, a inter mode SADgreater than FastIntraThreshold*N expected to correlate with a highprobability that a better intra mode SAD is possible. Thus program 600performs the calculation of intra mode SAD for the current macroblock atblock 608. Test block compares the best inter mode SAD from block 605with the best intra mode SAD from block 608. In accordance with theprior art, this comparison determines the encoding type for the currentmacroblock. If the best inter mode SAD is less than the best intra modeSAD (Yes at test block 609), then program 600 advances to block 607 tointer code the current macroblock. If the best inter mode SAD is notless than the best intra mode SAD (No at test block 609), then program600 advances to block 619 to intra code the current macroblock.

Following completion of inter coding in block 607 or intra coding inblock 610, test block 611 determines whether the current macroblock isthe last macroblock in the frame. If the current macroblock is not thelast macroblock in the frame (No at test block 611), the program 600advances to block 604 to consider the next macroblock. Program 600remains in this loop until the current macroblock is the last macroblock(Yes at test block 611).

If the current macroblock is the last macroblock in the frame (Yes attest block 611), then block 612 calculates FastIntraThreshold for thenext predictive frame. As noted above this calculation is based upon theaverage macroblock SAD and average variance in macroblock SAD of thecurrent frame. The average macroblock SAD value is:

$\begin{matrix}{{avg\_ sad} = {\frac{1}{M}{\sum\limits_{i = 1}^{M}\; {{sad}\lbrack i\rbrack}}}} & (1)\end{matrix}$

where: M is the number of macroblocks in the frame; and sad[i] is thecalculated inter mode SAD for the i th macroblock. The variance inmacroblock SAD is calculated as follows:

$\begin{matrix}{{var\_ sad} = {\frac{1}{M}{\sum\limits_{i = 1}^{M}\; \left( {{{sad}\lbrack i\rbrack} - {avg\_ sad}} \right)^{2}}}} & (2)\end{matrix}$

The actual computations are based upon average SAD per pixel andvariance per pixel calculated as follows:

$\begin{matrix}{{{avg\_ sad}{\_ pp}} = \frac{avg\_ sad}{N}} & (3) \\{{{var\_ sad}{\_ pp}} = \frac{var\_ sad}{N^{2}}} & (4)\end{matrix}$

where: N is the number of pixels per macroblock which is generally 256.The new value of FastInterThreshold for the next frame is:

FastIntraThrehold[n]=(1−QF*var_sad_pp[n−1])*avg_sad_pp[n−1]  (5)

where: QF is an empirical quality factor preferably selected from a setof 6 possible quality factor levels. The preferred values for the six QFlevels from 0 to 5 are shown in Table 3.

TABLE 2 Quality Factor Level FastIntraThreshold 0 INT_MAX 1 QF = 1/100 2QF = 2/100 3 QF = 3/100 4 QF = 4/100 5 0

A quality factor level 0 eliminates all intra mode SAD computations. Nomacroblocks are intra coded. This provides the lowest computationaloverhead and the least expected quality. For a quality factor level 5intra mode SAD computations occur for all macroblocks. This is expectedto yield the greatest video quality at the expense of the greatestcomputational overhead. Increasing the quality factor from 0 toward 5increases the percentage of macroblocks subject to intra mode SADcalculation. Such an increase in quality factor level is expected toincrease the computational overhead. Program 600 exits to the next framevia block 613 upon calculation of the next FastIntraThreshold.

In an H.264 encoder using a digital signal processor as described abovein conjunction with FIGS. 1 to 5 intra mode prediction generates around600 Bytes of DDR traffic for each macroblock. This amount of memorytraffic may prevent real time encoding at 30 frames per second (fps).This invention reduces DDR bandwidth by selectively performing intramode prediction. This invention makes a fast intra mode decisionalgorithm to accomplish this.

Experimental results show large video quality degradation on completeremoval of intra mode coded macroblocks. Selecting the quality factorlevel to eliminate 40% to 60% of the intra mode coded macroblocksresults in good combination of reduced computation and minimized videoquality degradation. The quality factor level 2 selects best tradeoff interms of video quality degradation and performance improvement. Thisavoids intra prediction estimation for about 55% macroblocks with verysmall degradation of 0.01 db on average and worst case degradation of0.04 db in perceived signal to noise ratio (PSNR) in the encoded video.

This invention predicts that intra mode prediction is more effective forthe macroblocks where motion estimation in inter mode prediction fails.This failure is indicated by a large value of the inter mode SAD. Thisinvention performs intra mode prediction for only macroblocks havinglarger inter mode SADs. The definition of a large inter mode SAD differsfor different content. This invention compares the inter mode SAD of acurrent macroblock with an adaptive threshold. This adaptive thresholddepends on the average and variance of the SADs of the previouspredicted frame. An adaptive threshold is calculated for each frame.

What is claimed is:
 1. A method for encoding video data having aplurality of frames, the method comprising: for each frame of theplurality of frames: dividing the respective frame into a plurality ofcoding units (CU) and, for each CU of the plurality of CUs: determiningan inter mode similarity measurement for the CU; comparing the intermode similarity measurement for the CU to a threshold determined basedon another inter mode similarity measurement corresponding to inter modepredictive encoding of a first frame that precedes the respective frameusing a second frame that precedes the first frame; if the inter modesimilarity measurement for the CU is less than the threshold, applyinginter mode predictive coding to encode the CU; and if the inter modesimilarity measurement for the CU is not less than the threshold, then:calculating an intra mode similarity measurement for the CU; comparingthe inter mode similarity measurement for the CU to the intra modesimilarity measurement for the CU; if the inter mode similaritymeasurement for the CU is less than the intra mode similaritymeasurement for the CU, applying inter mode predictive coding to encodethe CU; and if the inter mode similarity measurement for the CU is notless than the intra mode similarity measurement for the CU, applyingintra mode predictive coding to encode the CU.
 2. A method for encodingvideo data having a plurality of frames, the method comprising: for eachframe of the plurality of frames: dividing the frame into a plurality ofcoding units (CUs) and, for each CU of the plurality of CUs: determiningan inter mode similarity measurement for the CU; comparing the intermode similarity measurement for the CU to a threshold determined basedat least partially on inter mode similarity measurements of one or moreCUs of an immediately preceding frame, wherein the threshold isdetermined based on: an average of the inter mode similaritymeasurements of all CUs of the immediately preceding frame; and avariance of the inter mode similarity measurements of all CUs of theimmediately preceding frame; if the inter mode similarity measurementfor the CU is less than the threshold, applying inter mode predictivecoding to encode the CU; and if the inter mode similarity measurementfor the CU is not less than the threshold, then: calculating an intramode similarity measurement for the CU; comparing the inter modesimilarity measurement for the CU to the intra mode similaritymeasurement for the CU; if the inter mode similarity measurement for theCU is less than intra mode similarity measurement for the CU, applyinginter mode predictive coding to encode the CU; and if the inter modesimilarity measurement for the CU is not less than the intra modesimilarity measurement for the CU, applying intra mode predictive codingto encode the CU.
 3. The method of claim 1, wherein the inter modesimilarity measurement is a sum of absolute difference (SAD)measurement.
 4. The method of claim 2, wherein: the average of the intermode similarity measurements of all CUs of the immediately precedingframe is an average of inter mode sum of absolute difference (SAD)measurements for all CUs of the immediately preceding frame (avg_sad)determined as:${{avg\_ sad} = {\frac{1}{M}{\sum\limits_{i = 1}^{M}\; {{sad}\lbrack i\rbrack}}}},$wherein M is the number of all the CUs in the immediately precedingframe and sad[i] is the inter mode SAD measurement for the i-th CU ofthe immediately preceding frame; and the variance of the inter modesimilarity measurements of all CUs of the immediately preceding frame isa variance of the inter mode SAD measurements for all CUs of theimmediately preceding frame (var_sad) determined as:${var\_ sad} = {\frac{1}{M}{\sum\limits_{i = 1}^{M}\; {\left( {{{sad}\lbrack i\rbrack} - {avg\_ sad}} \right)^{2}.}}}$5. The method of claim 4, wherein the threshold is determined by:calculating an average inter mode SAD measurement per pixel for theimmediately preceding frame (avg_sad_pp) as${{{avg\_ sad}{\_ pp}} = \frac{avg\_ sad}{N}};$ and calculating avariance in inter mode SAD measurement per pixel for the immediatelypreceding frame (var_sad_pp) as${{{var\_ sad}{\_ pp}} = \frac{var\_ sad}{N^{2}}};$ wherein N is thenumber of pixels per CU of the immediately preceding frame, and whereinthe threshold is based on both avg_sad_pp for the immediately precedingframe and var_sad_pp for the immediately preceding frame.
 6. The methodof claim 5, wherein the threshold is determined as a product of a firstterm that includes the var_sad_pp for the immediately preceding frameand a second term equal to avg_sad_pp for the immediately precedingframe.
 7. The method of claim 6, wherein the first term further includesan empirical quality factor (QF).
 8. The method of claim 7, wherein thethreshold is calculated as equal to:(1−QF(var_sad_pp[n−1]))×(avg_sad_pp[n−1]), wherein n−1 is theimmediately preceding frame.
 9. The method of claim 8, wherein theempirical quality factor is selected from a set {0, 1, 2, 3, 4, 5},wherein, when the empirical quality factor is selected as 0, intra modecoding is not used for any CUs of the frame, and wherein, when theempirical quality factor is selected as 5, intra mode coding used forall CUs of the frame.
 10. A method of transforming an input video intoan encoded video, the method comprising: receiving a plurality ofsequential frames corresponding to the input video; dividing each frameinto a plurality of macroblocks; for each frame, coding each macroblockby: calculating an inter mode similarity measurement for inter modepredictive coding of the macroblock; comparing the inter mode similaritymeasurement with a threshold, wherein the threshold is determined as afunction of an average of inter mode similarity measurements ofmacroblocks of an immediately preceding frame and a variance of theinter mode similarity measurements of the macroblocks of the immediatelypreceding frame; if the inter mode similarity measurement is less thanthe threshold, then inter mode predictive coding the macroblock; and ifthe inter mode similarity measurement is not less than the threshold,then: calculating an intra mode similarity measurement for intra modepredictive coding of the macroblock; comparing the inter mode similaritymeasurement with the intra mode similarity measurement; if the intermode similarity measurement is less than the intra mode similaritymeasurement, then inter mode predictive coding the macroblock; and ifthe inter mode similarity measurement is not less than the intra modesimilarity measurement, then intra mode predictive coding themacroblock.
 11. The method of claim 10, comprising outputting the codedplurality of macroblocks of each frame as part of the encoded video. 12.The method of claim 10, wherein the inter mode similarity measurement isa sum of absolute difference (SAD) measurement.
 13. The method of claim12, wherein: the average of the inter mode similarity measurements ofmacroblocks of the immediately preceding frame is an average of intermode SAD measurements of macroblocks of the immediately preceding frame(avg_sad) determined as:${{avg\_ sad} = {\frac{1}{M}{\sum\limits_{i = 1}^{M}\; {{sad}\lbrack i\rbrack}}}},$M is a number of macroblocks in the immediately preceding frame andsad[i] is the calculated inter mode SAD measurement for the i-thmacroblock of the immediately preceding frame; and the variance of theinter mode similarity measurements of the macroblocks of the immediatelypreceding frame is a variance of the inter mode SAD measurements ofmacroblocks of the immediately preceding frame (var_sad) determined as:${var\_ sad} = {\frac{1}{M}{\sum\limits_{i = 1}^{M}\; {\left( {{{sad}\lbrack i\rbrack} - {avg\_ sad}} \right)^{2}.}}}$14. The method of claim 13, wherein the threshold is determined bycalculating an average inter mode SAD measurement per pixel for theimmediately preceding frame (avg_sad_pp) as${{{avg\_ sad}{\_ pp}} = \frac{avg\_ sad}{N}};$ calculating a variancein inter mode SAD measurement per pixel for the immediately precedingframe (var_sad_pp) as${{{var\_ sad}{\_ pp}} = \frac{var\_ sad}{N^{2}}};$ wherein N is thenumber of pixels per macroblock of the immediately preceding frame, andwherein the threshold is based on both avg_sad_pp for the immediatelypreceding frame and var_sad_pp for the immediately preceding frame. 15.The method of claim 14, wherein the threshold is determined as a productof a first term that includes the var_sad_pp for the immediatelypreceding frame and a second term equal to avg_sad_pp for theimmediately preceding frame.
 16. The method of claim 15, wherein thefirst term further includes an empirical quality factor (QF).
 17. Themethod of claim 16, wherein the threshold is calculated as equal to:(1−QF(var_sad_pp[n−1]))×(avg_sad_pp[n−1]), wherein n−1 is theimmediately preceding frame.
 18. The method of claim 16, wherein: theempirical quality factor is selected from a set {0, 1, 2, 3, 4, 5}; whenthe empirical quality factor is selected as 0, intra mode coding is notused for any macroblocks; and when the empirical quality factor isselected as 5, intra mode coding used for all macroblocks.